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ABSTRACT 

Holland House is a residential program for neglected 
and delinquent youth in Saginaw (Michigan). In 1992-93, the program 
received Education Consolidation and Improvement Act Chapter 1 
funding, which was used for materials and a teacher and an aiae for a 
reading skills tutorial program. This product evaluation was 
conducted in 1992-93, and focuses on the degree tc which participants 
achieve the major objectives of the program. Student entry and exit 
normal curve equivalent (NCE) scores on the Woodcock Johnson Tests of 
Achievement — Standard Battery were compared. Twenty-five students 
were served by the program in 1992-93 (14 Black, 9 White, and 2 
Hispanic) , and 11 students for whom there were complete data were 
considered in the study. The grant performance criterion specified 
that otudercs would improve more than three NCEs in reading. Test 
scores indicate that the program attained its objective in both basic 
and advanced reading. Seven of the 11 evidenced gains in both areas, 
1 experienced a gain in only 1 area, and 3 showed losses in both. 
Recommendations for program improvement are offered. Three tables 
present study findings. Four appendixes contain a weekly evaluation 
form and three additional tables. (SLD) 
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INTRODUCTION 

Holland House is a resid«nCial program designed for neglecCed and delin- 
quent youChs. During 1992-93, Che program received Education Consolidation 
and Improveinen t Act (ECIA) Chapter 1 funding. This funding was used toward 
the purchase of materials for a reading skills tutorial program and to 
reimburse the program for salary paid to a teacher and a part-time aide. As 
well, the supervisor and two staff members attended Chapter 1 sponsored 
inservice sessions. 

The majority of the instruction received by the students in this program 
occurred at Nouvel Central Catholic High School located in Saginaw Township. 
During the day, students attended classes in a classroom specifically set 
aside for the use of this program. Some students, depending upon their needs 
an{] abilities, also take classes in the regular Nouvel program. The remainder 
of the instrtiction occurred at the Holland House group home (614 East Holland 
Avenue, Saginaw, Michigan). During the evening, students attended additional 
sessions of study and instruction which varied from one to two hours in 
1 ength. 

Student progress was monitored on weekly evaluation sheets. Each student 
received a weekly description of his behnvior and participation and a letter 
grade reflecting his academic performance. Appendix A contains a blank copy 
of this evaluation form. 

The balance of the text of this report will present the procedure's and 
findings of the 1992-93 product review. A chart detailing the program's 1992- 
93 Chapter I expenditures can be found in Appendix B and a list of Chapter I 
inservice sessions attended by tlie staff is in Appendix C. 



PROCEDURES 



A product evaluation focuses on the degree to which participants achieve 
the major objectives of the program. This evaluation consisted of comparing 
students' entry and exit normal curve equivalent (NCE) scores on the Woodcock- 
Johnson Tests of Achievement - Standard Batte ry (WJ-R-ACH) Form A against the 
success criterion (specifically, an average gain in excess of three NCE's in 
basic and advanced reading scores). This test is not on the Michigan 
Department of Education's (MDOE) list of approved tests nor is it the test 
specified in the funding grant; however, it was nationally normed 1986-1988 
(Woodcock and Mather, 1989) and, according to the program supervisor, is 
widely used in the Michigan Boysville Program, 

PARTICIPANTS 

During 1992-93, 25 students were served by the program,^ Of these, 14 
(56,0"O were Black, 9 (36.0%) were White, and 2 (8.0%) were Hispanic. They 
ranged in age from 13 to 17 years. 

Fleven of the 25 (44.0%) program participants were considered in this 
study. Of the other 14, 9 (36.0%) entered the program with less than 5 months 
remaining in the school year, and 5 (20.0%) left the program without taking a 
pos t-tes t . 

METHOD 

In late August, 1992 (or upon entry to the program), the students were 
pre~tested and in May, 1993 (or just before exiting the program), the students 
were post-tested using the reading subtests of the WJ-R-ACH. 




^Although 25 students were served by the program throughout the year, 
there were .^t most 13 students in the program at one time. Students were 
released from and admitted to the program ov«='r the course of the year. 
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FINDINGS 

Student performance was examined to determine whether the program attain- 
ed its standard, 

ATTAINMENT OF THE STANDARD 

The performance criterion specified in the grant was that students would, 
in aggregate, evidence gains over and above the Michigan standard (more than 
three NCE's) in basic and advanced reading skills. Operationally, this was 
defined as growth in NCE units on, respectively, the broad reading and reading 
comprehension subtests of the WJ-R-ACH. 

Table 1, below, presents the average gains, in NCE's, in basic and 
advanced reading evidenced by the students during 1992-93, Table D-1, in 
Appendix D, presents the actual pre- and post-test NCE scores, and change, for 
each of the 11 students considered in the study. 

Table 1 

Attainment of the Performance Standard 





Mean gain 


Standard 


Reading skil 1 


in NCE's 


a t ta ined 


Broad reading (basic) 


1.1 


Yes 


Pass.^ge comprehension (advanced) 


3.5 


Yes 



Note. N=ll students. 

^Standard: Post-test NCh, scores will on average exceed three NCE's 
greater than pre-test scores. 



Bas^id upon the data contained in Table 1, it can be concluded that the 
program attained its objective in both basic and advanced reading * 

Also from Table D-l, one can find that 7 (63.6%) of the 11 students 
evidenced gains in both basic and advanced reading, I (9.1%) experienced a 
gain in basic but a loss in advanced reading, and 3 (27.3%) demonstrated 
losses in both reading areas. 
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SUMMARY 

During 1992-93, the Holland House program provided classroom and tutorial 
instruction as well as guidance and counseling to 25 youths^ 

The program received Chapter I funding to purchase Lns t rue t iona 1 
materials to help defray the salaries of a teacher and a part-time aide for a 
tutorial reading program* 

The aim of this evaluation review was to examine the amount of growth Ln 
reading skills evidenced by the participants* An average gain in excess of 
three normal curve equivalents (NCE) units Ln both basic and advanced skills 
as measured by the WJ-R-ACH was set as a standard for the program* This 
standard was attained * 

Beginning on the next page, recommendations will be offered* 
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RECOMMENDATIONS 



Prior to offering recommendations, it should be noted that the program is 
attaining its goals. It is providing a remedial program to delinquent youth, 
only one aim of which is to improve those students' reading skills. These 
recommencfa t ions are presented with the intention of enhancing an already 
effective program's approach to that one aim. 

It should also be noted that the recommendations below are not meant to 
be exhaustive; the enhancements they suggest may be attainable through other 
means. The supervisor and staff may want to consider what other means are 
available and should seek assistance from the district's Compensatory 
Education and/or Evaluation, Testing, and Research departments. 



• The staff should continue, where possible, to attend 
inservices offered through the Saginaw Public Schools' 
Instructional and Staff Development Center, Further, 
they 3hould maintain dialogue with local Chapter 1 
personnel. Both of these vehicles can provide a prof- 
itable interchange of ideas, 

• The supervi<^or should try to determine the reason for 
the success in student performance (whether it was 
due mostly to individual differences, program priori- 
ties [directly related to student need], instructional 
changes, or student attitude during the test) and what 
could be done to sustain positive per forma nee , 

• Student performance should be assessed by the test 
specified on the funding grant (e,g., the Hood cock 
Reading Mastery Tests - Revised, 1987, Forms G & H ) 
or the grant ohould be modified to specify the test 
used • 
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APPENDIX A 
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APPENDIX B 



DISBURSEMENT RECORD OF CHAPTER 1 FUNDS FOR 
THE HOLLAND HOUSE PROGRAM DURING 1 992-93. 



ITEM 



I 



Capital /Suppl les 
Sa 1 a r y Re i m b u r s em e n t 
Conferences 



COST 

10,408-00 
884.00 
412.00 



Tota 1 Expend 1 tare 



$ 11, 704,00 



These supplies included: 

• Supplemental and summer school textbooks 

• Resource books (e.g., encyclopedia, dictionaries) 

• Magazines and newspapers 

• Maps 

• Paperback books 

• Videos 

• Computer suppl ies 

• Shelves and office furniture 

• Testing materials. 

• Classroom supplies (e.g., pencils, pens, paper, folders, notebooks, 
and calculators). 



The total allocation for the program was $16,238.00, which left $4,534.00 
unspent as of the time this report was written. Plans for the expenditure 
of these funds during Summer, 1993, have already been made. 
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APPENDIX C 



Table C-1 

Chapter 1 Inservice Attendance 



Ins vice 



Staff Attqnding 
ABC Count 



Recognizing Tourette Syndroim in the classroom 
and in daily life situations. 

After school cross age tutoring. 

The changing role of Chapter 1 teachers and aides. 

Fostering self-esteem of children with LD or ADD 
from blame to empathy. 

Selected strategies for fostering self-esteem, mo- 
tivation and learning. 

Child abuse. 

Student emergencies in the classroom and on the 

playground. 
CAIS-conputers are tools. 

Teaching students to ask authentic questions. 

Chapter 1 math in the middle school. 

Active teaching for active learning in nath. 

ilake them laugh and they're yours. 

Managing resistant and oppositional behaviors. 

VJhy Johnm can't write. 

Answers to your questions. 

Reading, writing and self esteem. 

Cooperat ive . 1 earnings 

The effect of oculomotor and visual perceptive 
dysfunction on learning styles and academic 
achievements • 



X 
X 



X 
X 

X 
X 
X 
X 
X 
X 
X 
X 



X 
X 

X 
X 
X 
X 



1 
I 

3 
2 

2 
1 

1 
I 
1 
1 
2 

2 
1 
2 

I 
1 
1 
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APPENDIX D 



Table 

D-1 Broad Reading and Passage Comprehension Gains 

4 





Broad 


read Lng 


NCE' s 


Passage comprehe 


nsLon NCE's 


Student 


Pre 


Post 


Gain 


Pre 


Post 


Gain 


A 


44 


46 


2 


28 


35 


7 


B 


49 


42 


- 7 


52 


47 


- 7 


C 


49 


77 


28 


49 


63 


14 


D 


42 


46 


4 




j\) 


6 




\j 


36 


8 


38 


43 


5 


F 


25 


33 


8 


45 


42 


- 3 


G 


41 


45 


4 


34 


40 


6 


H 


15 


29 


14 


29 


34 


5 


I 


17 


10 


- 7 


26 


19 


- 7 


J 


35 


33 


- 2 


41 


36 


- 5 


K 


35 


62 


27 


42 


59 


17 


M 


34.5 


41.7 


7.2 


36.0 


42.5 


3.5 


(SD) 


(11.4)(16.7)(11.3) 


(1 1.6) 


(11.7) 


(7.7) 



Note* Maximum score = 100» N=l 1 students* 

^Broad reading and passage comprehension are conceptually equivalent to 
total reading and readin^^ comprehension, respectively, on otL^r norm- 
referenced standardized tests such as the California Achievement Tests* 
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